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1.0  Summary 


NEMSE’s  goal  is  to  facilitate  the  evaluation  and  maturation  of  fundamental  research  being 
conducted  within  the  AFOSR  Complex  Networks  program.  In  pursuit  of  this  goal  in  FY 1 1 
NEMSE  has  successfully: 

1)  Completed  its  investigation  of  network  emulation  techniques  for  the  Information 
Directorate’s  EMULAB; 

2)  Completed  the  selection  of  compatible  emulation  techniques  that  allows  working  with  all 
layers  of  the  Open  System  Interconnect  (OSI)  reference  model  network  stack; 

3)  Provided  a  simple  method  of  defining  EMULAB  experiments  in  PowerPoint  reducing  the 
learning  requirements  for  investigators; 

4)  Completed  eight  demonstrations  that  implement  the  emulation  environment; 

5)  Developed  three  virtualizations  for  FY12  continuation  that  contain  the  demonstrations. 

The  emulation  environment  is  available  in  FY12  for  use  by  complex  network  investigators 
under  NEMSE  funding  or  to  Information  Directorate  investigators  under  6. 2/6. 3  funding. 

2.0  Research  Objectives 

NEMSE’s  objective  is  to  facilitate  the  evaluation  and  maturation  of  fundamental  research 
being  conducted  within  the  Air  Force  Office  of  Scientific  Research  (AFOSR)  Complex 
Networks  program.  In  particular,  NEMSE  seeks  to: 

1)  Bridge  the  gap  between  basic  research  and  application  by  applying  tools,  techniques  and 
algorithms  to  scenarios  of  military  interest; 

2)  Serve  as  an  honest  broker  for  comparison  of  similar  tools  and  techniques; 

3)  Identify  paths  along  which  interesting  research  can  move  to  the  next  Research  and 
Development  (R&D)  level  within  Air  Force  Research  Laboratory  (AFRL)  or  elsewhere  in  the 
Department  of  Defense  (DoD). 

Additional  objectives  are  to  facilitate  and  encourage  the  use  of  valuable  modeling  resources, 
such  as  the  AFRL/Information  Directorate's  EMULAB  test-bed,  and  to  conduct  verification  and 
validation  (V&V)  of  research  models  or  hardware. 
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3.0  Technical  Summary 

Toward  the  goals  of  facilitating  the  use  of  the  Information  Directorate  EMULAB  and 
bridging  the  gap  between  basic  research  and  practical  application,  much  of  the  NEMSE  effort  in 
FY 1 1  consisted  of  developing  and  demonstrating  tools  and  techniques  intended  to  extend 
EMULAB’ s  functionality. 

3.1  EMULAB 

The  Information  Directorate  EMULAB  cluster  is  a  functional  copy  of  the  pioneering 
EMULAB  cluster  at  the  University  of  Utah.  While  there  are  at  least  37  DoD  and  University 
computer  centers  that  have  EMULAB  clusters,  only  the  Information  Directorate  EMULAB 
cluster  currently  has  the  NEMSE  extensions. 


The  EMULAB  cluster  was  designed  to  allow  realistic 
testing  of  network  designs  in  real-time  with  operators-in-the- 
loop.  Demonstrations  are  performed  on  a  designated  portion  of 
the  cluster  with  individual  users  located  at  their  separate  geo¬ 
locations.  An  interface  is  created  linking  the  operator's 
computer  with  the  cluster.  Each  operator  is  able  to  install  the 
software  of  his  assigned  computers  in  the  experimental  network 
and  act  as  the  administrator. 

Operational  hardware  or  a  prototype  can  be  incorporated  in 
a  specially  designed  NEMSE  box  and  placed  in  the  EMULAB  Figure  1 :  EMULAB 

cluster,  a  technique  called  hardware-in-the-loop  or  system-in-the  loop.  Standard  stand-alone 
emulation  packages  and  hardware  can  be  incorporated  into  a  node  inside  the  EMULAB  cluster. 


Experiential  networks  are  easy  to  set  up  in  EMULAB. 

Current  and  future  communication  technologies  can  be 
emulated  using  a  selected  set  of  commercial  and  DoD 
emulation  software  packages.  The  Information  Directorate 
EMULAB  has  been  enhanced  by  the  NEMSE  team.  This 
Environment  allows  incorporation  of  carefully  selected 
university,  commercial,  and  DoD  standard  emulation 
software  packages.  NEMSE  enhancements  have  been 
designed  to  provide  effective  support  for  both  operator-in- 
the-loop  and  hardware-in-the-loop. 

3.2  Demonstrations 

A  series  of  demonstrations  are  summarized  below  and  discussed  in  greater  detail  in 
Appendix  B.  The  demonstrations  have  served  to  improve  in-house  skills,  extend  the  capabilities 
of  the  Information  Directorate  EMULAB,  and  create  paths  for  further  research.  For  FY12,  the 
NEMSE  team  will  continue  to  work  with  the  Information  Dynamics  in  Networks  MURI  (Multi- 
University  Research  Initiative).  Before  expending  further  effort  on  other  EMULAB 


Figure  2:  FPLE  NEMSE  Box  installed  in 
EMULAB 
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demonstrations,  however,  the  NEMSE  team  will  seek  renewed  direction  from  the  Complex 
Networks  Program  Manager. 

3.2.1  Medium  Scale  Close  Air  Support  (CAS)  Demo 

This  emulated  network  demonstrates  basic  EMULAB  features  including  use  of  NS2 
(network  simulator)  agents  to  generate  traffic  at  specified  times.  This  demonstration  required  27 
physical  nodes  to  emulate  16  experimental  nodes  and  associated  links.  (The  extra  nodes  serve  as 
“delay  nodes”  used  to  implement  the  various  transmission  delays  specified  for  the  links).  It  is 
designed  to  represent  a  CAS  scenario,  including  ground  personnel,  unmanned  aerial  systems, 
satellites,  and  a  Tactical  Operations  Center  (TOC).  The  original  topology  was  provided  by 
Architecture  Technology  Company  (ATC)  Rome,  New  York  under  a  separate  contract  and 
modified  by  the  NEMSE  team.  We  also  added  a  network  testing  utility  from  the  Internet 
Performance  Working  Group  (IPERF)  to  each  node  to  facilitate  bandwidth  and  error  testing. 

3.2.2  System-in-the-Loop  Using  OPNET  Modeler  Demo 

OPNET  Modeler  is  a  highly-capable  network  simulation  tool.  Used  with  OPNET  System-in- 
the-Loop  (SITL)  and  EMULAB,  it  allows  researchers  to  combine  the  realism  of  the  emulated 
network  with  the  power  of  the  simulation  tool.  In  the  future,  we  anticipate  connecting  live  data 
streams  to  the  emulation  for  realistic  experiments  and  evaluations.  In  order  to  make  Modeler 
more  available,  we  installed  Modeler  on  the  EMULAB  cluster  such  that  any  authorized  user  can 
access  the  software  (with  concurrent  use  limited  by  the  number  of  licenses).  In  FY1 1,  the 
NEMSE  team  updated  the  FY10  demo. 

3.2.3  COPE  Demo 

COPE  is  a  forwarding  architecture  that  substantially  improves  the  throughput  of  wireless 
networks.  COPE  inserts  a  coding  shim  between  the  IP  and  MAC  layers,  which  identifies  coding 
opportunities  and  benefits  from  them  by  forwarding  multiple  packets  in  a  single  transmission. 
COPE  is  an  element  of  the  Information  Dynamics  in  Networks  MURI  (Multi-University 
Research  Initiative)  sponsored  by  AFOSR.  Our  initial  plan  was  to  install  and  test  COPE  on  the 
RI  EMULAB.  However,  COPE  is  inherently  wireless,  whereas  the  RI  EMULAB  does  not 
currently  have  wireless  nodes.  There  were  a  number  of  attempts  to  compensate  for  the  lack  of 
wireless  nodes  on  the  RI  EMULAB.  All  were  unsuccessful.  In  addition  the  team  had  to  deal 
with  the  restrictions  placed  on  non-citizens  when  they  access  a  DoD  computer  system.  The 
NEMSE  team  and  the  MURI  liaison,  Dr.  Markopoulou,  determined  the  best  course  of  act  was  to 
implement  COPE  on  the  Utah  EMULAB  and  funded  an  in-house  contractor  to  do  so.  That  effort 
is  underway.  Once  the  implantation  on  the  Utah  EMULAB  is  complete,  the  NEMSE  team  will 
assist  the  MURI  team  in  evaluating  COPE  modifications  and  reconsider  the  possibility  of 
implementing  it  on  the  RI  EMULAB.  For  background  information  on  COPE,  see  XORs  in  The 
Air:  Practical  Wireless  Network  Coding,  S.  Katti,  et  al .,SIGCOMM’06,  September,  2006,  Pisa, 
Italy. 
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3.2.4  FPGA  Physical  Layer  Emulator  (FPLE)  Demo 

The  Field  Programmable  Gate  Array  (FPGA)  Physical  Layer  Emulation  (FPLE) 
demonstration  is  the  implementation  of  a  concept  intended  to  both  extend  the  capabilities  of  the 
Information  Directorate  EMULAB  and  make  FPGA  design  more  accessible  to  the  researcher. 

The  FPLE  consists  of  an  Altium  Nanoboard  3000  installed  in  a  standard  rack  drawer.  There  are 
connections  to  allow  easy  use  at  a  lab  workstation  or  direct  physical  connection  to  the  EMULAB 
test-bed.  Use  of  Altium  Designer  allows  users  to  design,  compile  and  test  FPGA  code  without 
learning  VHDL  or  a  similar  FPGA  coding  language.  The  simple  physical  interface  allows  a  user 
to  easily  transfer  the  experiment  between  a  desktop  and  a  rack  in  the  EMULAB  test-bed.  In 
principle,  a  “NEMSE  box”  such  as  this  could  also  be  transferred  to  and  from  a  third  party,  such 
as  a  university  researcher,  for  research  in  an  outside  environment. 

3.2.5  GNU  Radio  Demo 

GNU  Radio  (GNU  is  not  Unix)  is  an  open-source  software  package  for  building  and  testing 
software-defined  radio  systems.  Used  in  connection  with  a  suitable  hardware  device,  such  as  the 
Universal  Software  Radio  Peripheral  (USRP),  GNU  Radio  provides  a  simple,  flexible  means  for 
emulating  a  variety  of  radio  devices.  As  with  the  FPLE,  the  demonstration,  consisting  in  this  case 
of  a  USRP  and  microcomputer  with  the  GNU  radio  software,  are  placed  in  a  NEMSE  box  for 
ease  of  transition  between  desktop,  EMULAB,  and  outside  researchers. 

3.2.6  Tech  Warrior  Demo 

The  Tech  Warrior  demonstration,  named  after  the  AFRL  exercise,  is  intended  to  represent  a 
field  scenario  with  airborne  surveillance  and  tactical  communication  between  multiple  ground 
units. 

3.2.7  CASCON  Demo 

The  CASCON  (Close  Air  Support  Connectivity)  demonstration,  springing  from  the  Close 
Air  Support  (CAS)-Connectivity  program,  was  a  war-gaming  scenario  intended  to  demonstrate 
and  test  video-based  CAS. 

3.2.8  RAVC  Demo 

The  Rate  Adaptive  Video  Coding  (RAVC)  demonstration  uses  a  commercial  video 
encoder,  the  FastVDO  SmartCapture  device,  to  control  the  transmission  (frame  rate  and 
resolution)  and  encoding  (compression)  characteristics  of  a  video  stream  to  adapt  to  changing 
bandwidth  limitations. 

3.3  Virtualizations 

The  eight  demonstrations  described  in  the  previous  sections  have  been  combined  into  three 
virtualizations  for  FY12.  A  NEMSE  virtualization  uses  real  processors  and  real  network 
components  but  the  network  itself  is  virtual. 
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These  virtualizations  are  described  in  Appendix  B.  “Evaluation  of  Complex  Network 
Abstract  Geometry”  (ECNAG). 


4.0  NEMSE  Elements 

The  NEMSE  elements  table,  Figure  3,  reconciles  the  various  elements  of  NEMSE  against 
the  OSI  stack  and  other  functions. 
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Figure  3:  NEMSE  Elements 


5.0  Conclusions 

The  software  environment  defined  by  the  NEMSE  team  is  complete  to  the  extent  that  is 
deemed  to  be  in  scope  of  the  NEMSE  project.  It  is  now  ready  for  use  by  AFOSR  researchers  to 
evaluate  Complex  Network  Portfolio  algorithms. 

Network  emulation  plays  an  important  role  in  evaluating  next-generation  complex  network 
product  designs.  From  the  component  level  to  the  system-of-systems  level,  emulation  enables 
evaluation  in  a  real  system  context.  Not  all  network  simulators  are  capable  of  true  emulation.  To 
be  a  true  replacement  for  a  network,  emulation  must  support  real-time  speed,  full  packet  fidelity, 
and  provide  transparency.  An  EMULAB  cluster  with  NEMSE  extensions  can  supports  true 
network  emulation  with  parallel  execution  and  the  highest  fidelity  models  and  the  scalability  and 
interactivity  required  to  test  and  evaluate  the  most  advanced  new  network  communications 
devices  and  architectures. 

Under  the  NEMSE  program,  complex  network  emulation  techniques  have  been  investigated; 
compatible  emulation  techniques  for  all  OSI  network  stack  layers  have  been  selected.  Other 
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significant  accomplishments  of  the  NEMSE  program  include  formulation  of  a  simple  method  of 
defining  EMULAB  experiments  in  PowerPoint  reducing  the  learning  requirements  for 
investigators,  the  completion  of  eight  demonstrations  that  implement  the  emulation  environment, 
and  the  development  of  three  virtualizations  for  FY12. 

The  NEMSE  EMULAB  emulation  environment  is  available  for  continued  use  during  FY12 
to  other  DoD,  industry,  and  academia  researchers  in  need  of  a  realistic  network  emulation 
environment. 

Appendix  A:  In-House  Activities 

Personnel: 


Name 

Degree 

Discipline 

Involvement 

Air  Force  Employees 

Dr.  David  Hench 

Ph.D. 

Electrical  Engineer 

1/5 

Capt  Craig  Baker 

M.S. 

Physicist 

1/5 

On-site  Contractors 

Ms.  Geraldine  Rogers 

B.S. 

Computer  Science 

1/5 

Mr.  Barry  Fabrey 

A.S. 

Automatic  Test  Equipment  Eng 

1/5 

Mr.  Chester  Wright 

M.S. 

Computer  Engineering 

1/20 

Appendix  B:  NEMSE  Demonstrations 


B  1  Introduction 

Rather  than  reinvent  the  wheel,  NEMSE  has  concentrated  on  making  a  modeling  and 
simulation  environment  where  the  best  of  current  emulation  techniques  are  available  to  users. 
NEMSE  is  now  available  in  an  easy-to-use,  integrated  form  on  the  Information  Directorate 
EMULAB.  FY 1 1  activities  concentrated  on  ensuring  that  protocol  development,  packet  statistics, 
and  transition  paths  are  available  for  all  levels  of  the  OSI  protocol  stack. 

The  eight  NEMSE  demonstrations  were  described  in  the  body  of  this  report.  They  have 
been  grouped  into  three  virtualizations  for  possible  continuation  in  FY12.  The  first  two 
virtualizations  are  for  continuation  of  the  Complex  Network  Portfolio  collaboration.  The  third 
virtualization  will  be  developed  using  6.2  research  funds.  These  three  virtualizations  are: 

1.  Evaluation  of  Complex  Network  Abstract  Geometry  (ECNAG):  Batch  mode  network 
characterization.  This  virtualization  includes  the  NEMSE  Medium  Scale  CAS  Demo.  The 
researcher  uses  a  methodology  for  designing  network  experiments  and  collecting  data  that  does 
not  require  a  Complex  Network  Investigator  to  know  the  details  of  EMULAB  or  even  to  sign  on 
to  EMULAB. 

2.  University  of  California,  Irvine  Collaboration  (UCIC):  Interaction  with  UC  Irvine 
including  part  of  the  COPE  Demo. 
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3.  Virtualization  of  Tech  Warrior  (VTWar)  which  transitions  the  NEMSE  Demos  to  6.2 
research.  The  NEMSE  Demos  were  System-in-the-Loop  (STIL)  using  OPNET  Modeler,  COPE, 
FPGA  Physical  Layer  Emulator  (FPLE),  GNU  Radio,  CORE  &  EMANE,  Tech  Warrior, 
CASCON  (CAS  Connectivity),  and  Rate  Adaptive  Video  Coding  (RAVC). 

The  common  network  models,  including  wireless,  are  provided  in  either  OPNET  or  in 
CORE  with  the  EMANE  class  library.  OPNET  is  mainly  known  as  a  simulation  product  but  with 
the  System-in-the-Loop  (SITL)  module  it  is  an  effective  emulation  package.  The  researcher  is 
able  to  feed  real  packets  in  and  receive  real  packets  out.  Multiple  instantiations  of  OPNET  are 
possible  in  a  single  experiment.  Using  a  combination  of  multiple  instantiations  of  OPNET  with 
multiple  instantiations  of  CORE  and  EMANE  along  with  hardware-in-the-loop  provides  a 
unique  capability  for  Verification  and  Validation  (V&V). 

OPNET  and  CORE  development  in  NEMSE  are  described  in  Section  B  3  along  with  other 
wireless  physical  layer  techniques  including  802. 1 1  emulations  using  Atheros  wireless  cards, 
GNU  radio,  and  an  FPGA  development  platform.  The  Tech  Warrior  virtualization,  Section  B  4, 
shows  how  OPNET,  CORE,  and  hardware-in-the-loop  can  be  combined. 

The  sign  on  procedure  for  the  Information  Directorate  EMULAB  was  developed  by 
University  of  Utah  and  adapted  by  the  Information  Directorate's  High  Performance  Computing 
group.  This  procedure  is  shown  in  Figure 

4.  The  steps  are: 


1 .  Get  a  Kerberos  (krb5)  Ticket. 

2.  Make  sure  X-Win32  is  enabled. 

3.  Get  a  Putty  window. 

4.  Type  "Firefox"  in  Putty  window. 

5.  Login  to  the  Project  Design  GUI. 


B  2  Evaluation  of  Complex  Network 
Abstract  Geometry  (ECNAG) 


B  2.1  Description  of  Batch  Mode 
Techniques 

NEMSE  Standard  Symbols  for 
Network  Drawings,  see  Figure  5,  shows 
a  standard  set  of  PowerPoint  symbols 
that  an  investigator  can  use  to  design  an 
experiment  for  EMULAB.  The  actual 
code  is  too  simple  to  justify  automating 
this  procedure  and  a  technician  can 
easily  transcribe  the  resulting  Network 
Drawings  into  an  EMULAB  experiment. 


Figure  4:  Log-On  Procedures  for  EMULAB 


Figure  5:  NEMSE  Standard  Symbols  for  Network  Drawings 
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The  Reduced  Set  of  Network  Symbols,  Figure  6,  can  be  used  by  a  researcher  who  desires 
to  work  problems  in  OSI  layers  only  above  the  MAC  sub-layer  of  the  Ethernet  Physical  Layer 
Protocol.  In  this  case,  the  native  Ethernet  EMULAB  hardware  can  be  used  instead  of  using  one 
of  the  available  NEMSE  physical  layer  emulation  techniques. 


With  the  approval  of  the  AFOSR  program 
manager,  the  Complex  Network  Investigator 
provides: 

1.  A  network  drawing,  Symbols  Figure  6. 

2.  Any  programs  or  shell  scripts  he  wants  to 
use. 

3.  A  measurement  template,  Figure  7 

4.  An  events  template,  Figure  8 

Then  the  NEMSE  team  can  provide  an 
EXCEL  spreadsheet  matching  the  measurement 
template.  This  technique  minimizes  the  learning 
curve  for  the  investigator.  As  a  hypothetical  example  assume 
a  Complex  Network  Investigator  wishes  to  investigate 
Routing  Protocols  in  the  Link  Layer.  Since  the  Link  Layer  is 
above  the  MAC  sub-layer,  the  Investigator  uses  the  reduced 
set  of  symbols.  Figure  7:  Event  Table  Template 


Time  (min) 

1  Event 

10 

Shutdown  Node5 

20 

Link  6  80%  packet  loss 

Figure  6:  Reduced  Set  of  Network  Symbols 


The  Network  Diagram  for  the  example  in  Figure  9  shows  the 
Network  Drawing  using  the  reduced  set  of  network  symbols. 
Each  processor  can  have  up  to  three  Ethernet  connections. 

The  Investigator  defines  various  links: 


Time  TCP  Ping 

(min)  Capacity  time 

Nodel-  Nodel- 

Node 10  Node  2 

(kbits/sec)  (sec) 


•  Ll 2  is  a  100  Mbps  high  quality  trunk. 

•  Ll 3  is  a  500  kbps  link  subject  to  natural 
interruption. 

•  L14isa56  kbps  link  subject  to  natural 
interruption. 

•  L24  is  a  200  kbps  link  subject  to  jamming. 

•  L23  is  a  100  Mbps  link  subject  to  jamming. 

•  L34  is  a  100  Mbps  link  subject  to  jamming. 

It  is  assumed  that  jamming  has  the  following 
level  of  risk: 


Figure  8:  Sample  Measurement 
Template 


N2 


N3 


Figure  9:  Network  Diagram  for  the  Example 


•  Jamming  link  L34  is  low  risk. 

•  Jamming  links  L34  and  L23  or  link  L24  is  medium  risk. 

•  Jamming  links  L34,  L23,  and  L24  is  high  risk. 


Approved  for  Public  Release ;  Distribution  Unlimited. 
8 


The  Investigator  decides  to  use  an  IPERF  server  in  node  N1  using  Fedora  12  nodes  and  an 
IPERF  client  in  node  N2  to  measure  the  capacity  in  kbps.  In  this  example  the  Investigator  wants: 

1.  An  initial  run  using  the  router  built  into  Fedora  12. 

2.  Jamming  to  be  either  full  on  or  full  off. 

3.  Capacity  measurements  every  10  seconds. 

4.  Data  delivered  as  concatenated  IPERF  readings. 

5.  Five  minutes  between  events  to  ensure  routing  has 
settled  down. 

6.  The  event  table,  Figure  10. 


Status:  This  example  is  currently  in  an  advanced  sate 
of  debugging. 

B  2.2  NEMSE  Medium  Scale  CAS  Demo 

During  FY10,  a  20-node  tactical  network  developed  by  ATC  New  York  for  EMUFAB  was 
studied  and  exercised.  This  study  included  signing  onto  various  nodes  and  becoming  familiar 
with  the  programming,  including  the  use  of  NS2  agents.  This  model  was  demonstrated  to 
AFOSR.  The  models  used  in  this  network  were  in  need  of  more  accurate  development. 

During  FY1 1  this  model  was  transformed  into  a  batch  experiment  with  Fedora  12  nodes 
and  IPERF  in  each  node.  A  Wireshark  network  analyzer  was  also  integrated. 

It  is  considered  that  this  demo  puts  the  team  in  a  good  position  to  gather  statistics  to 
support  geometric  models  from  the  AFOSR  Complex  Networks  Portfolio. 

B  3  University  of  California  Collaboration  (UCIC) 

UC  Irvine  provided  two  techniques  to  be  considered  by  NEMSE  team.  These  two 
techniques  are  the  Click  router  and  the  MadWiFi  driver.  The  COPE  experiment  provides  a 
network  coding  shim  between  the  Fink  Fayer  and  the  MAC  sub  layer  of  the  Physical  Fayer. 

B  3.1  Click 

Click  is  a  software  architecture  for  building  flexible  and  configurable  routers  that  was 
developed  at  MIT.  A  Click  router  is  assembled  from  packet  processing  modules  called  elements. 
Individual  elements  implement  simple  router  functions  like  packet  classification,  queuing, 
scheduling,  and  interfacing  with  network  devices. 

Users  can  program  their  own  elements  in  C++  allowing  greater  flexibility.  Alternately,  a 
less  sophisticated  user  can  use  preprogrammed  elements. 

Complete  configurations  are  built  by  connecting  elements  into  a  graph;  packets  flow  along 
the  graph’s  edges. 
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Figure  10:  Event  List  for  the  Example 
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A  typical  Click  router  is  meant  to  be  used  in  the  Linux  kernel  module.  This  means  that  a 
hardware  implementation  of  the  Click-developed  router  is  a  simple  embedded  Linux  module 
with  processing  power  matched  to  measured  requirements  in  the  emulation. 


B  3.2  MadWiFi 

The  MadWiFi  driver  is  an  open  source  802.11 
Linux  driver  with  well  documented  source  code.  A 
proprietary  Hardware  Abstraction  Layer  (HAL)  in 
binary  code  is  available  for  MadWiFi  for  the 
Atheros  AR52 1 0,  AR52 1 1 ,  AR52 12,  RF5111, 

RF51 12,  RF2413,  and  RF5413  wireless  cards  with 
capability  for  PCI,  miniPCI,  and  PC  card  bus 
interfaces.  There  are  no  known  USB  HALs.  Since 
MadWiFi  is  an  actual  Linux  driver,  emulation  can 
be  ported  to  a  Linux  embedded  system  with  an 
Atheros  card.  The  Atheros  cards  are  part  of  the 

University  of  Utah  EMULAB  and  the  RF  physical-layer  hardware  emulator  designed  for 
NEMSE  will  be  shown  in  the  next  section. 

A  work  flow  diagram  of  the  NEMSE  COPE 
project  is  shown  in  the  work  flow  for  porting 
COPE  Figure  1 1 .  The  source  code  for  COPE  was 
provided  by  UC  Irvine  from  the  graduate  student 
project.  Since  a  foreign  graduate  will  be 
continuing  the  research,  the  Information 
Directorate  reinstalled  the  source  code  on  the  U  of 
Utah  EMULAB. 

The  RF  Channel  Emulator,  Figure  12,  uses 
two  ASUS  PCE-N13  802.1 1  b/g/n  Wireless  PCI  - 
Express  Adapter  cards  plus  two  fixed  and  two 
variable  attenuators,  a  splitter/mixer,  and  rigid  coax  cables.  This  produces  an  analog  RF  channel 
emulator  that  is  complete  except  for  delay  elements  which  can  either  be  incorporated  in  analog 
or  in  digital  in  a  modified  MadWiFi  adaptor.  The  connector  and  cable  losses  of  this  emulator  can 
be  calibrated  in  the  RF  laboratory  using  standard  techniques. 


Figure  12:  RF  Channel  Emulator 


Figure  1 1 :  Work  Flow  for  Porting  COPE 


If  a  Hardware  Abstraction  Layer  was  written  for  the  FPLE  (Section  B  4.5  below),  the 
Information  Directorate  would  have  the  capability  to  emulate  802.1 1  Link  and  Physical  Layer 
using  either  RF  or  FPLE  emulation  of  the  transmission. 
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B  4  Virtualization  of  Tech  Warrior  (VTWar) 

B  4.1  Tech  Warrior  exercise 

Tech  Warrior  is  an  AFRL  exercise  that  provides  AFRL  Company  Grade  Officers  who  have 
little  or  no  operational  experience  with  an  intense  field,  mobility,  and  combat  skills  training 
opportunity.  Tech  Warrior  also  provides  AFRL  the  opportunity  to  test,  experiment,  conduct  data 
collection,  and  or  otherwise  showcase/demonstrate  state-of-the-art  war  fighting  technologies  in  a 
realistic  operational  environment. 

The  Small  Unmanned  Systems  Exploitation  (SUSEX),  RYRA  program,  was  participating 
in  the  Tech  Warrior  exercise.  They  were  conducting  flight  testing  with  a  variety  of  small 
unmanned  platforms.  These  flights  collected  data,  demonstrate  sensor  exploitation  capabilities, 
and  develop  operations  concepts  using  small  UASs  and  unattended  ground  sensors  in  support  of 
intelligence,  surveillance,  reconnaissance,  and  targeting  activities. 

The  NEMSE  team  viewed  SUSEX  participation  in  Tech  Warrior  as  an  excellent 
opportunity  to  collect  and  process  numerous  video  feeds  in  support  of  the  continuing 
development  of  NEMSE.  In  addition  to  the  video  recorded,  SUSEX  provided  us  with  a  copy  of 
their  data  collection  files;  13,879  files,  79  GBytes.  The  data  classification  was  FOUO  and  for 
limited  distribution.  We  can  use  the  data  internal  to  RI,  but  we  are  unable  to  share  it  with 
AFOSR  Complex  Networks  researchers.  This  inability  to  share  the  collected  data  with  AFOSR 
researchers  was  a  major  disappointment. 

The  following  sensor  payloads  were  flown  by  SUSEX: 

Gimbaled  IR  Video  -  LWIR,  MWIR,  &  SWIR,  WAMI  -  1-5  frame/sec,  large  footprint, 

Gimbaled  EO  Video  -  26x  zoom,  AR  -  1  ft  Imagery,  Targeting,  Change  Detection,  and 

SIGINT  -  Demodulation,  Limited  Direction  Finding. 

The  picture  on  the  right  started  out  as  a  very  high  definition 
video  feed  from  a  SUAS  to  the  SUSEX  control  van,  over  a  44 
Mbps  link.  The  communication  link  from  the  control  van  to  the 
FOB  was  a  10  Mbps  link.  The  video  feed  at  the  FOB  was  junk. 

The  picture  on  the  right  is  excellent  example  for  making  the  case 
for  more  research  by  AFOSR  in  the  area  of  reliable  transfer  of 
data  in  dynamic,  hostile  and  high  interference  environments. 

NEMSE  considers  the  Tech  Warrior  Exercise  to  be  a  success  because  the  data  collected 
enable  us  to  build  the  Tech  Warrior  Virtualization.  This  virtualization  can  potentially  be  used  to 
design  the  FY12  Tech  Warrior  Exercises  with  enhanced  capability. 

B  4.2  Tech  Warrior  Virtualization 

The  Network  drawing  of  the  Tech  Warrior  Demo,  Figure  13,  shows  the  network  drawing  of 
the  Tech  Warrior  Virtualization.  This  drawing  uses  the  symbols  of  Reduced  Set  of  Network 
Symbols  and  is  produced  by  the  investigator  in  PowerPoint. 
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The  script  file  that  implements  this 
virtualization  is  shown  below  in  its  entirety  to 
illustrate  the  reason  that  it  was  not  considered 
time  effective  to  automate  the  procedure.  Such  a 
script  is  relatively  easy  for  a  trained  programmer 
to  produce. 

#  Start  of  script 
set  ns  [new  Simulator] 
source  tb  compat.tcl 

#  Allocate  nodes 
set  DataRelay  [$ns  node] 
tb-set-node-os  $DataRelay  WinXP- 

Modelerl5-Svr 

set  AOC2  [$ns  node] 
tb-set-node-os  $AOC2  WINXP-VLC 
set  AOC1  [$ns  node] 
tb-set-node-os  $AOCl  WINXP-VLC 
set  AOC3  [$ns  node] 
tb-set-node-os  $AOC3  WINXP-VLC 
$ns  make-lan  "$AOCl  $AOC2  $AOC3  $DataRelay"  100Mb  0ms 
set  FOB3  [$ns  node] 
tb-set-node-os  $FOB3  WINXP-VLC 
set  FOB2  [$ns  node] 
tb-set-node-os  $FOB2  WINXP-VLC 
set  FOB  1  [Snsnode] 
tb-set-node-os  $FOBl  WINXP-VLC 

$ns  make-lan  "$FOBl  $FOB2  $FOB3  $DataRelay"  100Mb  0ms 

set  UAS  [$ns  node] 

tb-set-node-os  $UAS  FBSD73-STD 

set  TOC  [$ns  node] 

tb-set-node-os  $TOC  TOC 

set  CommandLink  [$ns  duplex-link  $AOC2  $UAS  10Mb  0ms  DropTail] 
set  ROVERlink  [$ns  duplex-link  $UAS  $TOC  10Mb  0ms  DropTail] 
#tb-set-hardware  $TOC  pc3000 

#  Start! 

$ns  run 


FOB1...3 

Figure  13:  Network  drawing 
Tech  Warrior  demo 
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Figure  14,  EMULAB  Visualization  of  Tech  Warrior,  shows  the  visualization  produced  by 
EMULAB  from  the  above  script.  OPNET  will  be  described  in  Section  B  4.3  OPNET,  GNU 
Radio  in  Section  B  4.4,  FPGA  Physical  Layer  Emulator  (FPLE)  in  Section  B  4.5,  CASCON  in 
Section  B  4.6,  and  RAVC  in  Section  B  4.7. 
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Figure  14:  EMULAB  Visualizai 

don  of  Tech  Warrior  Demo 

B  4.3  OPNET 

OPNET  Technologies,  Inc.  provides  performance  analysis  for  computer  networks  and 
applications.  OPNET  products  address  application  performance  management,  network 
performance  management,  and  network  R&D.  OPNET  mixes  Discrete  Event  Simulation  (DES) 
and  statistical  flows.  With  its  Wireless  for  Defense  package,  OPNET  has  excellent  mobility  and 
antenna  modeling  capability.  OPNET  is 
generally  considered  a  modeling 
package  with  simulated  packets  in  its 
DES  simulation.  However,  with  its 
System-in-the-Loop  (SITL)  models  it 
becomes  a  real-time  emulation  package 
with  real  packets  input  to  the  computers 
Network  Interface  Card  (NIC)  and  real 
packets  out  of  another  NIC.  OPNET 
has  excellent  packet  data  collection  and 
statistical  analysis  packages.  OPNET, 
however,  is  not  a  source  of  real  packets. 


The  FY10  final  report  described 
OPNET  work  in  a  demonstration  called 
OpPhy  that  modeled  a  jammer  scenario 

Figure  15:  Antenna  Pattern  in  OpNet 
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with  antenna  models  as  a  variable  component  for  anti-jam  resistance.  Figure  15  repeats  an 
antenna  pattern  that  was  taken  from  measurements  in  the  Information  Directorate  Anechoic 
chamber. 


In  addition  to  mobility  and  antenna  modeling,  the  OPNET  installation  allows  physical  level 
modeling  through  a  series  of  pipeline  stages  with  the  C++  code  in  each  stage  open  for 
modification  with  well  documented  interface  structures. 


Six  stages  on  the  transmitter  side  are: 

Receiver  group 
Transmission  delay 
Link  closure 
Channel  match 
Tx  antenna  gain 
Propagation  delay 


Eight  stages  on  the  Receiver  side  are 

Rx  antenna  gain 
Received  power 
Interference  noise 
Background  noise 
Signal-to-noise  ratio 
Bit  error  rate 
Error  allocation 
Error  correction 


The  OPNET  demonstration  has  been  placed  in  the  VTWar  Data  Relay  node. 

Status :  OPNET  is  currently  fully  supported  and  detailed  modeling  is  out  of  scope  for  the 
NEMSE  program. 


B  4.4  GNU  Radio 


GNU  Radio  is  a  free  software  development  toolkit  that  provides  the  signal  processing 
runtime  and  processing  blocks  to  implement  software  radios  using  readily-available,  low-cost 
external  RF  hardware  and  commodity  processors.  All  of  the  code  is  under  copyright,  by  the  Free 
Software  Foundation.  It  is  widely  used  in  hobbyist,  academic,  and  commercial  environments  to 
support  wireless  communications  research  as  well  as  to  implement  real-world  radio  systems. 

GNU  Radio  applications  are  primarily  written  using  the  Python  programming  language, 
while  the  supplied,  performance-critical  signal  processing  path  is  implemented  in  C++  using 
processor  floating  point  extensions  where  available.  Thus,  the  developer  is  able  to  implement 
real-time,  high-throughput  radio  systems  in  a  simple-to-use,  rapid-application-development 
environment. 


While  not  primarily  a  simulation  tool,  GNU  Radio  does  support  development  of  signal 
processing  algorithms  using  pre-recorded  or  generated  data,  avoiding  the  need  for  actual  RF 
hardware. 


The  Universal  Software  Radio  Peripheral™  (USRP)  product  family  allows  you  to  create  a 
software  defined  radio  using  any  computer  with  a  USB  2.0  or  Gigabit  Ethernet  port.  Various 
plug-on  daughterboards  allow  the  USRP  and  USRP2  SDRs  to  be  used  on  different  radio 
frequency  bands.  Daughterboards  are  available  from  DC  to  5.9  GHz  at  this  time.  The  entire 
design  of  the  USRP  family  is  open  source. 
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The  USRP  product  family  works  with  GNU  Radio,  a  tree-software  (open  source) 
framework  for  the  creation  of  software  defined  radios.  GNU  Radio  works  on  the  FreeBSD 
operating  system  such  as  is  used  in  the  VTWar  UAS  node. 

The  NEMSE  Box  for  GNU  with  NTSC  ROVER  Camera,  Figure  16,  shows  a  NEMSE  box 
constructed  to  house  up  to  two  USRP  2  radios.  At 
the  present  time  there  is  only  one  radio  installed. 

The  camera  shown  by  the  side  was  built  to 
calibrate  ROVER  IV  units  in  the  field  and  a 
USRP,  GNU  radio  pair  has  been  used  to  decode 
NTSC  analog  video  that  could  be  displayed  by  a 
ROVER.  The  built-in  processor  can  run  an 
Access  10  component  such  as  those  shown  in  a 
RF  channel  emulator,  Figure  12. 


B  4.5  FPGA  Physical  Layer  Emulator  (FPLE) 

This  demo  is  currently  a  standalone  experiment  that  can  be  used  to  link  the  VTWar  A0C1 
node  to  the  UAS  node. 

The  FPLE  objectives  were: 

1 .  Prototype  an  FPGA  interconnect  between  two  computers  that  can  be  used  to  emulate 

physical  layer  and  channel  effects. 

2.  Integrate  an  FPGA  development  and  testing  environment  into  the  EMULAB  cluster  to 

allow  simple  design  by  any  Information  Directorate  engineer. 

3.  Develop  C  language  Mix  drivers  for  MATLAB. 

4.  Investigate  methodology  for  FPGA  interconnection  between  computers. 

An  effective  and  simple  FPLE  has  been  demonstrated  that  utilizes  the  Altium  Nanoboard 
3000  hardware  and  Altium  designer  software.  This  combination  provides  an  integrated  hardware 
development  environment,  FPGA  hardware  interfaces,  and  testing  for  hardware  and  software  in 
MATLAB.  The  hardware  provides  access  to  most  standard  computer  interfaces. 

Graphical  development  using  standard  electrical  engineering  techniques  avoids  use  of 
layout  languages  such  as  VHTML.  These  techniques  make  FPGA  design  accessible  to  any 
electrical  engineer. 

The  FPLE  has  been  integrated  into  a  2U  high  standard  rack  for  potential  use  in  the 
EMULAB  cluster.  USB  connections  from  each  of  two  computers  can  be  used.  A  simple 
prototyping  area  is  included. 

Use  of  IP  cores  is  emphasized.  The  cores  used  may  be  built-in  to  Altium  Designer, 
obtained  from  a  third  party,  or  developed  in-house  with  the  techniques  built  into  Altium 


Figure  16:  NEMSE  Box  for  GNU  with 
NTSC  ROVER  Camera 
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Designer.  Cores  for  many  logic  components  are  included  as  are  cores  for  memory,  processors 
and  DSPs,  and  the  Wishbone  bus  interconnects. 


The  software  diagram  for  FPLE,  Figure  17,  illustrates  the  software  concept.  The  FPLE 
utilizes  two  Digital  EO  modules  (USB-DIO-32  by  Access).  These  modules  are  the  same  as  the 
module  used  in  the  GNU  radio  variable  attenuator  controller  RF  channel  emulator,  Figure  12. 
These  devices  were  selected  primarily  because  of  the  flexibility  of  the  driver  software.  This 
flexibility  has  allowed  it  to  include  a  MEX  C  function  and  can  be  compiled  into  MATLAB  as  a 
C  language  extension. 


Figure  17:  Software  Diagram  for  FPLE 


Figure  18  shows  the  FPLE  NEMSE  box  mounted  in  the 
Information  Directorate  EMULAB.  A  standard  19”  x  21”  rack 
tray,  with  ball  bearing  sliders,  is  utilized  to  mount  the  required 
components.  This  tray  assembly  is  designed  to  be  quick- 
disconnect  so  that  it  may  also  be  used  on  a  lab  bench  or  in 
another  rack  oriented  system.  The  FPLE  tray’s  purpose  is  to 
provide  a  flexible  development  platform  for  engineers  and 
designers  of  FPGA  based  systems.  The  FPLE  tray  contains  the 
following  items: 


Figure  18:  FPLE  NEMSE  Box 
Installed  in  EMULAB 


1 .  Altium  NB3000XN  Nanoboard  Development  System,  used  to  develop  FPGA 
functional  code  on  a  Xilinx  3  AN  platform. 

2.  Cosel  R25A-5-N  five  (5)  volt  power  supply.  This  unit  supplies  5  volt  DC  power 
to  the  on  board  units,  eliminating  the  use  of  several  modular  120vAC/5vDC  plug  units  (for  each 
tray  item). 

3.  Access  EO  Products  USB-DIO-32  (2  each)  Digital  Serial  /  Parallel  10  unit.  One 
unit  connects  the  “A”  user  computer  USB  port  to  the  “A”  data  parallel  input  to  the  NB3000 
system.  The  other  unit  connects  the  “B”  data  parallel  output  of  the  NB3000  system  to  the  “B” 
user  computer  USB  port.  Power  for  these  units  is  supplied  via  the  USB  connection. 

4.  USB  Hub  (generic),  used  to  connect  the  “A”  computer  to  the  “A”  DIO  unit  and 
the  NB3000  development  system  (for  JTAG  down  loading  of  FPGA  configuration  files). 

5.  Break-Out  Board,  (in-house  design/build).  This  facilitates  the  use  of  a  Data 
Analyzer  in  the  parallel  data  paths  between  the  DIO  and  the  NB3000  data  paths  (A  and  B). 

6.  Rear  Panel  Assembly.  This  is  a  2U  panel  at  the  rear  of  the  tray.  On  it  is  located 
the  Power  Entry  Module,  with  switch  and  the  “A”  and  “B”  USB  panel  mount  connectors. 
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Figure  19,  shows  the  FPLE  NEMSE  box 
ruggedized  and  placed  in  the  prototype  CASCON  rack. 
This  box  was  hard  wired  and  used  for  NEMSE 
development  before  the  EMULAB  was  available. 

B  4.6  CASCON 

CASCON  was  a  series  of  field  experiments 
preceding  Tech  Warrior.  Photos,  videos,  maps,  and 
ConOps  were  accumulated  to  develop  a  War  Game 
Virtualization.  The  CASCON  work  has  been  joined  with 
the  Tech  Warrior  virtualization. 

B  4.7  Rate  Adaptive  Video  Coding 

This  demo  is  hardware  connected  between 
VTWar  Nodes  TOC  and  UAS.  Figure  20  shows 
the  hardware  diagram  with  USB  video  converted 
to  NTSC  (analog)  video  and  then  back  to  USB 
video.  Figure  21  shows  the  hardware  that  actually 
converts  the  monitor  video  to  analog  and  then  uses  the 
FastVDO  Universal  Video  Stick  to  create  RAVC  video. 

B  5  Conclusions 

The  software  environment  defined  by  the  NEMSE 
team  is  complete  to  the  extent  that  is  deemed  in  scope  of 
NEMSE.  It  is  now  ready  for  either  universities  for 
collecting  data  to  evaluate  Complex  Network  Portfolio 
algorithms  or  for  6. 2/6. 3  funded  programs. 


Figure  19:  FPLE  in  CASCON  Box 


Figure  20:  RAVC  Network  drawing 


Figure  21:  Hardware  for  RAVC 
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ACRONYMS 


MURI 

Multi- University  Research 

Initiative 

NEMSE 

Network  Modeling,  Simulation 
and  Emulation 

NETWARS 

Network  Warfare  Simulation 

NIC 

Network  Interface  Card 

NRL 

Naval  Research  Lab 

NS2 

Network  Simulator  2 

OPNET 

Optimized  Network  Evaluation 
Tool 

OSI 

Open  System  Interconnect 

PI 

Principle  Investigator 

R&D 

Ressearch  and  Development 

RAVC 

Rate  Adaptive  Video  Coding 

RDT&E 

Research,  Development  Test  & 
Engineering 

RF 

Radio  Frequency 

ROVER 

Remotely  Operated  Video 

Enhanced  Receiver 

SITL 

System-  in-  the-  Loop 

TCL 

Tool  Command  Language 

TOC 

Tactical  Operations  Center 

TPP 

Technology,  Practices,  and 
Procedures 

UAS 

Unmanned  Aerial  System 

UCIC 

University  of  California  Irvine 
Collaboration 

UGV 

Unmanned  Ground  Vehicle 

USRP 

Universal  Software  Radio 
Peripheral 

V&V 

Verification  and  Validation 

VHDL 

Very  High-level  Design  Language 

VTWar 

Virtual  Tech  Warrior 

WiMax 

Worldwide  Interoperability  for 
Microwave  Access 

AFOSR 

Air  Force  Office  of  Scientific 

Research 

AFRL 

Air  Force  Research  Lab 

API 

Application  Programmer  Interface 

ATC 

Architecture  Technology 

Company 

CAS 

Close  Air  Support 

CASCON 

Close  Air  Support  Connectivity 

ConOPS 

Concept  of  Operations 

CORE 

Common  Open  Research 
Environment 

DES 

Discrete  Event  Simulation 

DoD 

Department  of  Defense 

DSN 

Defense  Switched  Network 

ECNAG 

Evaluation  of  Complex  Network 
Abstract  Geometry 

EMANE 

Extensible  Mobile  Ad-hoc 

Network  Emulator 

FPGA 

Field  Programmed  Gate  Array 

FPLE 

(FPGA)  Physical  Layer  Emulation 

FPLE 

FPGA  Physical  Layer  Emulator 

FY 

Fiscal  Year 

GNU 

(GNU’s  not  Unix) 

GUI 

Graphical  User  Interface 

HARBS 

Hierarchical  AFRL  Research  for 
Battlefield  Support 

HPCMO 

High  Performance  Computing 
Management  Office 

IPERF 

Internet  Performance  Working 
Group 

JCSS 

Joint  Communications  Support 
Squadren 

MAC 

Media  Access  Control 

MURI 

Multi- University  Research 

Initiative 

NEMSE 

Network  Modeling,  Simulation 
and  Emulation 

NETWARS 

Network  Warfare  Simulation 

NIC 

Network  Interface  Card 
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